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(54) Image pickup apparatus and method of controlling the apparatus 



(57) An image of an object outside a vehicle is 
picked up by a left camera (11) and a right camera (12) 
under the control of a CPU (71) based on current posi- 
tion information (42) of the vehicle whose position is 
measured, vehicle state information (44) of slow speed, 



running, etc., and road information (31) concerning the 
road where the vehicle is positioned, such as a narrow 
street, and is supplied through a selector (22) to a dis- 
play (21 ), so that the need for the user to perform bur- 
densome work is eliminated and the user can check the 
circumstances outside the vehicle. 
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Description 

[0001] This invention relates to an image pickup ap- 
paratus installed in a vehicle and a control method of 
the image pickup apparatus, 

2. Description of the Related Art 

[0002] A system for picking up an image of the circum- 
stances surrounding a vehicle by a camera installed in 
the vehicle and displaying the picked- up image on a dis- 
play placed in the vehicle is developed. 
[0003] The user feels the inconvenience of manually 
performing various types of control such as starting and 
ending image pickup by a camera and the like. 
[0004] The present invention has been made to solve 
the above problem. 

[0005] To achieve the above object, according to one 
aspect of the Invention, there is provided an image pick- 
up apparatus comprising an image pickup section for 
picking up an image of an object outside a vehicle; a 
position detector for detecting a vehicle position; an in- 
formation storage section storing road information con- 
cerning roads; and a controller for controlling the image 
pickup section based on the road information and the 
vehicle position. 

[0006] According to another aspect of the invention, 
there is provided an image pickup apparatus comprising 
an image pickup section for picking up an image of an 
object outside a vehicle; a position detectorfor detecting 
a vehicle position ; a state detectorfor detecting a vehicle 
state; an information storage section storing road infor- 
mation concerning roads; and a controller for controlling 
the image pickup section based on the road information 
and the vehicle position. 

[0007] According to another aspect of the invention, 
there is provided an image pickup apparatus comprising 
an image pickup for picking up an image of an object 
outside a vehicle; a position detectorfor detecting a ve- 
hicle position; a state detector for detecting a vehicle 
state; an information storage section storing road infor- 
m ation concerning roads; a nd a controller for control li ng 
the image pickup based on the road information, the ve- 
hicle position, and the vehicle state. 
[0008] According to another aspect of the invention, 
there is provided an image pickup apparatus control 
method comprising a road information acquisition step 
of acquiring road information concerning a road where 
a vehicle is positioned; a run state informatbn acquisi- 
tion step of acquiring state information indicating a run 
state of the vehicle; and a control step of controlling im- 
age pickup means based on the road information and 
the state information from the steps. 
[0009] These and other objects and advantages of 
this invention will become more fully apparent from the 
following detailed description taken with the accompa- 
nying drawings fn which: 



FIG. 1 is a block diagram to show an embodiment 
of the invention; 

FIG, 2 is a block diagram according to an example 
of the invention; and 
s FIG, 3 is a flowchart according to the example of 
the invention. 

[001 0] Ref errm g now to the accompanying drawings, 
there is shown a preferred embodiment of the invention. 

10 [0011] FIG. 1 is a block diagram to show an embodi- 
ment of an image pickup apparatus according to the in- 
vention. In the figure, numeral 1 denotes image pickup 
means for picking up an image of the circumstances of 
the outside of a vehicle as an object, numeral 2 denotes 

w display means for displaying the image picked up by the 
image pickup means, numeral 3 denotes information 
storage means storing road information concerning 
roads f numeral 4 denotes position detection means for 
detecting the vehicle position, numeral 5 denotes state 

20 detection means for detecting the vehicle state, numeral 
6 denotes operation means for receiving user's opera- 
tion, and numeral 7 denotes control means for acquiring 
information from the means and controlling the opera- 
tion of the means. 

25 [0012] The information storage means 3 stores as the 
road information concerning roads, known shape infor- 
mation indicating the shapes of the roads defined by link 
information and node information and known type infor- 
mation indicating the types of roads classified according 

30 to the types of roads such as narrow street, prefectural 
road, and national road in association with the link infor- 
mation. 

[0013] The control means 7 determines the type of 
road where the vehicle is positioned at the current point 

35 in time from the type information acquired from the in- 
formation storage means 3 and vehicle position infor- 
mation indicating the current position of the vehicle ac- 
quired from the position detection means 4. The control 
means 7 also determines the vehicle position on the 

40 road, for example, the road positron of the vehicle on the 
road such that w the vehicle Is positioned XX meters be- 
fore an intersection" or that "the vehicle is positioned in 
the intersection" from the shape information acquired 
from the information storage means 3 and the vehicle 

& position information. 

[001 4] The state detection means 5 detects the vehi- 
cle state such as the run state of the traveling speed, 
the traveling direction, etc., of the vehicle, and gener- 
ates vehicle state information indicating the run state. 

so The control means 7 can determine the run state of the 
vehicle based on the vehicle state information. 
[0015] The operation means 6 comprises an opera- 
tion section for the user to set image pickup enable (on) 
or disable (off) of the image pickup means 1 and an op- 

55 eration section to set display on or off of the display 
means 2 and outputs an operation signal responsive to 
the actual operation of the user to the control means 7. 
[0016] According to the described configuration, the 
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control means 7 controls the image pickup means 1 or 
the display means 2 based on the road information of 
the road type, etc, and the vehicle position information 
of the road position, etc., based on the road Information 
and the vehicle state information of the run state, etc., 
and further based on the road Information, the vehicle 
position information, and the vehicle state information. 
[001 7] An example according to the embodiment will 
be discussed in detail with reference to FIGS, 2 and 3. 
[0018] FIG, 2 is a block diagram to show an example 
of the image pickup apparatus. In the figure, numerals 
11 and 12 denote cameras placed at the front of the ve- 
hicle; numeral 11 denotes a left camera for picking up 
an image of the circumstances at the left front of the 
vehicle as an object and numeral 12 denotes a right 
camera for picking up an image of the circumstances at 
the right front of the vehicle as an object. Various types 
of operation such as image pickup start and termination, 
angle , zoom, etc + , of each camera are controlled by a 
CPU 71. 

[0019] Numeral 21 denotes a display for displaying 
the image of the circumstances outside the vehicle 
picked up by each camera, a map image provided by a 
navigation system, a television image, a text information 
image received by a tuner, etc. Numeral 22 denotes a 
selector for selectively supplying one or a plurality of im- 
age signals to draw each image to the display 21 and 
the selection operation is controlled by the CPU 71 1 
[0020] Numeral 31 denotes a ROM disk recording 
various pieces of information that can be used with a 
known navigation system , such as map information in- 
cluding road information and facility information and 
service information including telephone numbers and 
zip codes. Numeral 32 denotes a read section for read- 
ing information from the ROM disk 31 under the control 
of the CPU 71 . Numeral 33 denotes RAM for temporarily 
storing information readby the read section 32. The 
RAM 33 also includes a video storage section for storing 
a map image signal sent to the selector 22; when It f u no- 
tions as a part of the navigation system, a map image 
based on the map image signal is displayed on the dis- 
play 21. 

[0021] Further, the ROM disk 31 records as road in- 
formation concerning roads, known shape information 
indicating the shapes of the roads defined by link infor- 
mation and node information and known type informa- 
tion indicating the types of roads classified according to 
the types of roads such as narrow street, prefectural 
road, and national road in the format corresponding to 
the link information. Further, the road information in- 
cludes shape information and type information defining 
a multilevel parking lot and an underground parking lot; 
areas into and from which a vehicle goes and exits are 
defined, 

[0022] Numeral 41 denotes a positioning section for 
measuring the current position of the vehicle. Numeral 
42 denotes a GPS section. The GPS section 42 is a 
known GPS section comprising a GPS antenna for ac- 



quiring a signal from a GPS satellite, a demodulator for 
demodulating the acquired signal, a computing unit for 
computing the demodulated data to find latitude infor- 
mation, longitude Information, and height information, 
5 and the like, and outputs the absolute position of the 
vehicle consisting of the latitude information, the longi- 
tude information, and the height information to the po- 
sitioning section 41. 

[0023] Numeral 43 denotes a gyro for detecting the 
10 rotation angle on the horizontal plan e of the veh icle and 
outputting relative angle information of the vehicle 
based on the rotation angle to the positioning section 
41 , Numeral 44 denotes a speed sensor for detecting 
speed pulses used, for example, to drive a speed meter 
*s in the vehicle and control anABS (anti-lock brake 
system) t and outputting the speed pulses to the posi- 
tioning section 41 . The positioning section 41 calculates 
relative position information to the reference position of 
the vehicle based on the relative angle information from 
20 the gyro 43 and the speed pulses from the speed sensor 
44. 

[0024] The positioning section 41 measures the cur- 
rent vehicle position {the current position of the vehicle) 
from the relative position information and the absolute 

25 position information acquired from the GPS section 42, 
Such a positioning method generally is called hybrid po- 
sitioning method and is a known positioning method. 
[0025] Numeral 51 denotes a speed detection section 
for detecting the period of the speed pulse detected by 

30 the speed sensor 44 and calculating the speed of the 
vehicfe. 

[0026] Numeral 61 denotes an operation section for 
receiving user 1 s operation; the operation section 61 is 
made up of a remote control, a touch panel, voice input 
35 means, etc, 

[0027] Numeral 71 denotes the above-mentioned 
CPU for receiving input of signals and information from 
the sections and controlling the operation of the sec- 
tions. 

40 [0028] NumeralSt denotesatunenltlsaknowntuner 
comprising a TV tuner and a radio tuner. 
[0029] The operation of the image pickup apparatus 
comprising the means will be discussed based on a 
flowchart of FIG. 3. 

45 [0030] To begin with, the control process for making 
a transition from a state in which neither an image of the 
left camera 11 nor an image of the right camera 12 is 
displayed on the display 21 {for example, a state in 
which a map is displayed) to a state in which the image 

50 are displayed will be discussed. 

[0031] As the state in which neither an image of the 
left camera 11 nor an image of the right camera 12 is 
displayed on the display 21 , a state in which the CPU 
71 controls the selector 22 for supplying and displaying 

55 a map image from the RAM 33 to and on the display 21 
can be taken as an example; it corresponds to the case 
of functioning as the navigation system, etc. In this state, 
at step $1 ( the CPU 71 determines that a camera image 
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is not selected (No at step S1) and goes to step S2, 
[0032] Step 32 is a step for determining the state of 
the vehicle based on the speed information of the vehi- 
cle. The CPU 71 , which acquires the speed information 
of the vehicle from the speed detection section 51 } de- 
termines whether or not the traveling speed of the vehi- 
cle at the point in time is predetermined speed or less 
at step $2* 

[0033] That is, the CPU 71 determines whether or not 
the state is "slow speed" state as an example of the ve- 
hicle state based on the traveling speed of the vehicle, 
To determine whether or not the state is "slow speed" 
state, the predetermined speed used as the reference 
speed Is previously defined and if the traveling speed of 
the vehicle calculated by the speed detection section 51 
is the predetermined speed or less (Yes at step S2), the 
CPU 71 determines that the state of the vehicle is "slow 
speed 1 ' and goes to step S3; if the traveling speed of the 
vehicle is more than the predetermined speed (No at 
step 82), the CPU 71 determines that the state of the 
vehicle is not "s low speed" and repeats processi n g start- 
ing at stepSL 

[0034] Step 3 is a step f or determ ining wheth er or not 
the type of road where the vehicle is positioned is a nar- 
row street from the current vehicle positionmeasuredby 
the positioning section 41 and the road type information 
read from the ROM disk 31 . That is, it is a step for de- 
termining the type of road where the vehicle is posi- 
tioned based on the current vehicle position and the 
road information concerning the road, 
[0035] If the vehicle is positioned on a narrow street 
(Yes at step S3), the CPU 71 goes to step S4 and de- 
termines whether or not the vehicle is positioned within 
a predetermined distance to an intersection, namely, 
whether or not the vehicle is about to enter the intersec- 
tion from the current position of the vehicle measured 
by the positioning section 41 and the road shape infor- 
mation read from the ROM disk 31 . 
[0036] If the vehicle is positioned within the predeter- 
mined distance to the intersection (Yes at step S4), the 
CPU 71 goes to step S5 and starts image pickup of ei- 
ther of the left camera 11 and the right camera 12 and 
controls the selector 22 for selecting an image signal 
from either of the left camera 11 and the right camera 
12, whereby the circumstances outside the vehicle are 
displayed on the display 21. 
[0037] The image pickup start timing of each camera 
depends on the value of the predetermined distance. 
That is, assuming that the predetermined distance is 
1 00 n% for example, image pickup of the camera is start- 
ed 100 m before the vehicle runs toward the intersec- 
tion, and the circumstances outside the vehicle as an 
object are displayed on the display 21 ♦ Assuming that 
the predetermined distance is 10 m> displaying the cir- 
cumstances outside the vehicle is started 10 m before 
the intersection, The value of the predetermined dis- 
tance may be previously defined by the manufacturer of 
the image pickup apparatus or may be able to be select- 



ed by the user. 

[0038] Thus t as it is determined that the state of the 
vehicle is "slow speed," that the vehicle is positioned on 
a narrow street, and that the vehicle is positioned within 
5 the predetermined distance to an intersection, the cir- 
cumstances outside the vehicle are displayed on the 
display 21 . 

[0039] Thus, when the vehicle is in the "slow speed" 
state and is about to enter the intersection on the narrow 

10 street, automatically the circumstances outside the ve- 
hicle are displayed on the display 21, eliminating the 
need for the user to perform burdensome work. 
[0040] A modification ot control for automatically 
switching into a camera image described above is as 

15 follows: 

[0041] A step of "is connection road at intersection 
narrow street? 1 ' is inserted between steps 84 and S5. 
That is, it is a step for determining whether the intersec- 
tion that the vehicle is about to enter is an intersection 

20 with a narrow street or an intersection with a principal 
street wider than a narrow street, such as a prefectural 
road oranationai road. Accordingly, whether the vehicle 
is about to enter a small intersection of narrow streets 
or is about to merge with or cross a principal street can 

£s be determined. 

[0042] The relationship between the determination re- 
sult at the additional step and the transition to step 35 
is determined appropriately, whereby the condition for 
switching into a camera image can be determined. That 

30 is, if it is determined that if Yes is returned from the ad- 
ditronal step H is connection road at intersection narrow 
street?," the transition to step S5 is made, it is made 
possible to switch into a camera image if the vehicle is 
about to enter a small intersection of narrow streets , On 

ss the other hand, if it is determined that if No is returned 
from the additional step "is connection road at intersec- 
tion narrow street?," it is made possible to switch into a 
camera image if the vehicle is about to merge with or 
cross a principal street from a narrow street Such a de- 

40 termination may be previously defined by the manufac- 
turer of the image pickup apparatus or may be abie to 
be selected by the user. 

[0043] As another modification, if step S5 is a step of 
"select camera image in response to traveling direction 
45 of vehicle," the image of the o bject provided by either o r 
both of the left camera 11 and the right camera 12 can 
be checked on the display 21 . That is, if the vehicle is 
about to make a left turn at an intersection, the CPU 71 
can controi the selector 22 for selecting the image signal 
50 from the left camera 11 to display the object image on 
the whole display 21 ; if the vehicle is about to make a 
right turn at an intersection, the CPU 71 can control the 
selector 22 for selecting the image signal from the right 
camera 12 to display the object image on the whole dis- 
ss play 21 ; and if the vehicle is about to travel in a straight 
line at an intersection, the CPU 71 can control the se- 
lector 22 for selecting the image signals from both the 
left camera 11 and the right camera 12 to display the 
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object image from the left camera 11 on the left half of 
the display 21 and the object image from the right cam- 
era 1 2 on the right half of the display 21 . 
[0044] Next, the control process for making a transi- 
tion from a state in which an image from either of the left 
camera 11 and the right camera 12 is displayed on the 
display 21 to a state in which the image is not displayed 
will be discussed. This is a state in which the display is 
restored to map display, etc., for example. 
[0045] At step S1 , the CPU 71 determines whether or 
not a camera image is selected as the sef ector 22 is con- 
trolled. If the CPU 71 determines that a camera image 
is selected (Yes at step S1), it goes to step S6, 
[0046] At Step 6, the CPU 71 determines whether or 
not the type of road where the vehicle is positioned Is a 
narrow street from the current position of the vehicle 
measured by the positioning section 41 and the road 
type Information react from the ROM disk 31 . It is a step 
for determining the type of road where the vehicle is po- 
sitioned based on the current position of the vehicle and 
the road information concerning the road. 
[0047] If the CPU 71 determines that the vehicle is po- 
sitioned on a narrow street (Yes at step S6), the CPU 
71 goes to step S7 and determines whether or not the 
vehicle Is positioned within a predetermined distance to 
an intersection, namely, whether or not the vehicle is go- 
ing away from the intersection from the current position 
of the vehicle measured by the positioning section 41 
and the road shape information read from the ROM disk 
31 

[0048] If the vehicle passes through the intersection, 
the CPU 71 determines that the vehicle is positioned 
within the predetermined distance to the intersection 
{Yes at step 56), in which case the CPU 71 returns. Ac- 
cordingly, selection of the camera image is continued 
and the circumstances outside the vehicle are continu- 
ously displayed on the display 21 . 
[0049] On the other hand, if the vehicle has passed 
through the intersection and is going away from the in- 
tersection, the CPU 71 determines thatthe vehicle is not 
positioned within the predetermined distance to the in- 
tersection (No at step S6), and goes to step S8. if the 
CPU 71 determines at step S6 that the vehicle is not 
positioned on a narrow street (No at step 56), for exam- 
ple, when the vehicle starts to run on a principal street, 
etc, the CPU 71 also goes to step S8. 
[0050J At step S8 T the CPU 71 controls the selector 
22 for releasing selection of the image signal from either 
of the left camera 11 and the right camera 12 and se- 
lecting the previous image signal selected just before 
the camera image was selected to display the image on 
the display 21, 

[0051] Thus, as it is determined thatthe vehicle is not 
positioned on a narrow street and that the vehicle is not 
positioned within the predetermined distance to an in- 
tersection, the display 21 is switched from the state in 
which the circumstances outside the vehicle are dis- 
played to the previous image. Thus, when the vehicle is 



going away from the intersection, automatically the dis- 
play is switched from the image of the circumstances 
outside the vehicle into a map image, for example, elim- 
inating the need for the user to perform burdensome 
5 work. 

[0052] In the described example, the image pickup 
apparatus comprises the positioning section 41 for de- 
tecting the vehicle position based on the information 
from the GPS section 42, the gyro 43, and the speed 

10 sensor 44, the speed detection section 51 for supplying 
the speed information for the CPU 71 to detect the state 
of the vehicle (such as slow speed), and the ROM disk 
31 storing the road Information concerning roads such 
as the road type information and the road shape infor- 

'5 mation, wherein an image of an object outside the vehi- 
cle is picked up by the left camera 1 1 and the right cam- 
era 12 under the control of the CPU 71 based on the 
road type information, the road shape information, the 
vehicle position information, and the vehicle state infor- 

20 mation, and is supplied through the selector 22 to the 
display 21 , so that the need for the user to perform bur- 
densome work is eliminated, 
[005S] In the described example, all conditions of the 
road ^formation, the vehicle position, and the vehicle 

25 state are determined and then Image pickup start of the 
camera is controlled, but various changes may be add- 
ed so that image pickup start of the camera is controlled 
immediately after the conditions of at least the road in- 
formation and the veh icle position are dete mnln ed or that 

30 image pickup start of the camera is controlled after the 
conditions of at least the road information and the vehi- 
cle state are determined. 

[0054] In the described example, approach to and go- 
ing away from the intersection point of roads such as 

35 the intersection point of a narrow street and a principal 
street asforthe intersection havebeendescribed. Asde- 
scribedearlier, the road information includes the shape 
information and the type information defining a parking 
lot and using the shape information and the type infor- 

40 mation , a point where the vehicle enters a road from the 
parking lot or a point where the vehicle enters the park- 
ing lot from a road is contained in the determination ob- 
ject at step S4 in FIG, 3, whereby it is made possible to 
control so as to select a camera image when the vehicle 

45 enters the parking lot or when the vehicle enters a con- 
nection road from the parking lot. 
[0055] The foregoing description of the preferred em- 
bodiments of the invention has been presented for pur- 
poses of Illustration and description. It is not intended to 

bo be exhaustive or to limit the invention to the precise form 
disclosed, and modifications and variations are possible 
in light of the above teachings or may be acquired from 
practice of the invention. The embodiments were cho- 
sen and described In order to explain the principles of 

55 the invention and its practical application to enable one 
skilled in the art to utilize the invention in various em- 
bodiments and with various modifications as are suited 
to the particular use contemplated, it is intended that the 
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scope of the invention be defined by the claims append- 
ed hereto, and their equivalents. 



Claims 

1 . An image pickup apparatus comprising: 

an image pickup adapted to pick up an image 

of an object outside a vehicle; 

a position detector adapted (o detect a vehicle 

position; 

an information storage adapted to store road 
information concerning roads; and 
a controller adapted to control said image pick- 
up based on the road information and the vehi- 
cle position. 

2. An image pickup apparatus comprising: 

an image pickup adapted to pick up an image 

of an object outside a vehicle; 

a position detector adapted to detect a vehicle 

position; 

a state detector adapted to detect a vehicle 
state; 

an information storage adapted to store road 
information concerning roads; and 
a controller adapted to control said image pick- 
up based on the road information and the vehi- 
cle state. 

3. An Image pickup apparatus comprising: 

an image pickup adapted to pick up an image 

of an object outside a vehicle; 

a position detector adapted to detect a vehicle 

position; 

a state detector adapted to detect a vehicle 
state; 

an Information storage adapted to store road 
Information concerning roads; and 
a controller adapted to control said image pick- 
up based on the road information, the vehicle 
position and the vehicle state. 

4. The image pickup apparatus as claimed in claim 1 
wherein the road information is obtained from said 
information storage based on the vehicle position, 

5. The image pickup apparatus as claimed in cfaim 4 
wherein the road information includes road informa- 
tion concerning a road where the vehicie is posi- 
tioned and road information concerning a road con- 
nected to the road where the vehicle is positioned. 

6. The image pickup apparatus as claimed in daim 2 
wherein said state detector comprises a speed de- 



tection section for detecting traveling speed of the 
vehicle, and 

wherein said controller determines whether or 
not the vehicle state is a run state based on the 
5 speed Information from the speed detection sec- 
tion, and controls said image pickup apparatus. 

7. An image pickup apparatus control method com- 
prising: 

10 

a road information acquisition step of acquiring 
road information concerning a road where a ve- 
hicle is positioned; 

a run state information acquisition step of ac- 
15 quiring state information indicating a run state 

of the vehicle; and 

a control step of oontroilin g image pickup based 
on the road information and the state informa- 
tion from said steps. 
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